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PREFACE 


This  report  was  created  for  the  F-16  Aircrew  Training  De¬ 
velopment  Project  contract  no.  F02604-79-C8875  for  the  Tactical 
Air  Command  to  comply  with  the  requirements  of  CDRL  no.  B049. 

The  project  entailed  the  design  and  development  of  an  instruc¬ 
tional  system  for  the  F-16  RTU  and  instructor  pilots.  During  the 
course  of  the  project,  a  series  of  development  reports  was  issued 
describing  processes  and  products.  A  list  of  those  reports 
follows  this  page.  The  user  is  referred  to  Report  No.  34,  A 
Users  Guide  to  the  F-16  Training  Development  Reports,  for  an 
overview  and  explanation  of  the  series,  and  Report  No.  35,  F-16 
Final  Report,  for  an  overview  of  the  Instructional  System 
Development  Project. 
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EXECUTIVE  SUMMARY 


*This  report  details  the  rationale  and  procedure  for 
sequencing  (syllabus  building)  the  pilot  and  instructor  pilot 
(IP)  syllabi  of  the  F-16.  Principles  of  instructional  sequenc¬ 
ing  are  presented  and  descriptions  given  of  the  F-16  sequencing 
process  and  syllabus  revision  procedures.^ 

Development  of  a  syllabus  involves  evaluation  of  (1)  student 
entry  level  skills,  (2)  training  and  instructional  goals/objec¬ 
tives,  (3)  availability  of  training  resources,  and  (4)  the  behav¬ 
ior  expected  of  graduates.  The  development  of  a  syllabus  is 
based  upon  a  set  of  learning  principles.  Three  examples  are: 

1.  Speed  of  forgetting  (forgetting  begins  almost  immediately 
unless  there  is  prompt  testing  or  rapid  use  of  new 
information) . 

2.  Interference  (learning  of  some  tasks  interferes  with  the 
learning  of  other  tasks  when  they  possess  similar  but 
confusing  differences.  Remembering  that  final  approach 
speeds  of  155,  140,  and  150  kts  are  characteristic  of  the 
F-4,  F-16,  and  A-7  respectively  is  more  difficult  than 
remembering  final  approach  speeds  of  155  and  60  for  the  F-4 
and  piper  cub  respectively) . 

3.  Transfer  (prior  learning  of  some  skills  enhances  the 
learning  of  another  skill.  Skill  at  proper  scanning  of 
instruments  for  IFR  flight  gained  in  one  aircraft  will 
facilitate  the  same  task  in  another  aircraft)  . 

The  syllabus  structure  must  be  linked  to  the  criterion- 
referenced  objectives  as  well  as  the  career  performance  goals  of 
the  students.  To  generate  a  syllabus  a  model  must  exist  which 
describes  the  necessary  steps  to  acquire  a  skill.  For  the  F-16 
this  involves: 

1.  Academic  training:  Base  preparation  for  task  execution 
(workbooks,  tape/slides,  videotapes,  lectures,  discussion 
groups,  and  tutorials) . 

2.  Practice  in  limited  simulated  performance  environment: 
Simulators,  mockups,  CPT,  dead  aircraft. 

3.  Practice  in  real  world  performance  environment  or  near  real 
world  environment.  OFT,  WST,  actual  aircraft. 


vii 
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Specific  procedures  for  generating  the  F-16  syllabus  involve 

1.  Definition  of  the  mastery  models  and  terminal  performance 
tests . 

2.  Determination  of  the  desired  levels  of  performance  for  each 
task  (given  by  the  CRTs) . 

3.  Determination  of  the  syllabus  structure  based  upon  aircraft 
as  the  only  training  device. 

4.  Determination  of  non-aircraft  training  devices  to  preserve 
aircraft  utilization. 
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F-16  INSTRUCTIONAL  SEQUENCING 
PLAN  REPORT 


1.0  INTRODUCTION 


This  report  contains  the  rationale  and  procedure  for  sylla¬ 
bus  building,  also  called  sequencing,  for  pilot  and  instructor 
pilot  syllabi  for  F-16.  It  attempts  to  present  information  rele¬ 
vant  to  a  varied  audience,  and  in  doing  so  covers  an  overview  of 
the  principles  of  instructional  sequencing  to  give  a  discussion 
context,  a  description  of  the  F-16  sequencing  process,  and  a 
description  of  syllabus  revision  procedures.  It  is  hoped  that 
those  responsible  for  managing  the  F-16  instructional  system, 
those  who  operate  it,  and  those  who  maintain  it  will  find  infor¬ 
mation  appropriate  to  their  needs  in  this  report. 


1.1  Organization 

Section  2.0  of  this  report  presents  an  overview  of  sequenc¬ 
ing,  its  relationship  to  other  instructional  development  proc¬ 
esses,  and  the  principles  it  attempts  to  incorporate  into  train¬ 
ing  syllabi.  Desirable  characteristics  of  syllabi  are  also 
dicussed. 

Section  3.0  describes  the  relation  of  syllabus  building  to 
the  instructional  technology  principle  of  criterion-referenced 
instruction  and  testing. 

Section  4.0  presents  a  model  of  skill  development  which 
serves  as  the  basic  building  block  of  the  F-16  syllabus.  This 
discussion  also  describes  the  full  context  of  instruction,  begin¬ 
ning  with  first  academic  exposure  to  instructional  material  and 
concluding  with  mastery  of  a  skill  executed  in  job-like  environ¬ 
ments  . 

Section  5.0  describes  syllabus  building  and  emphasizes  it  as 
a  generative  process  based  mainly  on  training  requirements  and 
limited  by  resource  availability  factors.  In  this  section  a 
discussion  of  syllabus  maintenance  highlights  the  importance  of 
continual  maintenance  of  the  syllabus  and  change  using  systematic 
procedures. 

Section  6.0  contains  a  step-by-step  description  of  the  F-16 
sequencing  procedure. 
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2.0  OVERVIEW  OF  SEQUENCING 


A  syllabus  designates  the  instructional  events,  both  aca¬ 
demic  and  training  device-related,  to  be  contained  in  a  course  of 
instruction  and  the  order (s)  in  which  they  may  or  must  be 
encountered  by  students. 

The  syllabus-building  process  is  one  of  maximizing  instruc¬ 
tional  and  performance  outcomes  of  a  course  of  instruction.  It 
should  be  performed  as  a  generative  process  in  which  the  raw 
materials  produced  during  job  and  hierarchical  analysis  are 
transformed  into  an  instructional  sequence.  Its  primary  aim  is 
to  define  a  sequence  of  instructional  events  which  promotes  a 
smooth,  stepwise  building  of  skills,  from  simple  basic  to  more 
complex  and  lengthy  skills,  for  a  given  population  of  students. 

The  syllabus-building  process  is  also  one  of  minimizing  the 
requirement  for  expenditure  of  training  resources  and  time.  The 
media  selection  plan  report  (project  report  no.  20,  "F-16  Media 
Selection  and  Utilization  Plan  Report")  describes  media  selection 
as  one  of  the  development  processes  most  influential  on  the  total 
cost  of  aircrew  training.  Syllabus  building  must  share  that 
title  because  it  determines  the  level  and  type  of  use  that 
instructional  media  devices  receive,  including  expensive  simula¬ 
tors  and  aircraft.  The  syllabus  also  determines  to  some  extent 
the  amount  of  maintance  and  support  effort  which  will  be  required 
to  conduct  training,  since  it  designates  aircraft  configurations 
needed  to  carry  out  particular  training  missions.  Fiinally,  it 
designates  the  level  of  use  of  instructional  personnel.  In  an 
activity  as  costly  as  aircrew  training  and  particularly  for  one 
carried  on  the  scale  of  F-16  training,  it  can  be  seen  that  small 
adjustments  in  either  the  syllabus  or  the  media  selections  will 
have  a  profound  effect  on  the  total  training  expense.  This 
increases  the  desirability  of  systematic,  precise  methods  of 
syllabus  generation. 

Inherent  in  a  given  syllabus  are  certain  assumptions  about 
preparation  level  of  incoming  students,  the  expected  output  level 
of  students  graduated  from  the  course,  the  training  resources 
available,  including  instructors,  materials,  equipment,  and 
facilities.  For  a  given  set  of  incoming  student  capabilities  and 
graduate  expectations,  a  particular  syllabus  should  represent  the 
best  use  of  training  resources.  There  is  an  interdependent  rela¬ 
tionship  between  these  factors,  and  the  change  of  one  occasions  a 
change  in  the  others.  Figure  1  illustrates  this  balance.  As  the 
student  population  changes,  or  if  graduate  expectations  change, 
or  if  the  resources  devoted  to  training  change,  the  syllabus 
itself  must  change  also. 
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Figure  1  -  Balance  of  Factors  in  Syllabus  Construction 
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2.1  Relation  of  Sequencing  to  Other  Instructional  Development 

Processes 

Sequencing  has  direct  relationships  with  almost  every  other 
process  in  instructional  development.  Prior  processes  produce 
the  elements  and  constraints  for  syllabus  development,  and  subse¬ 
quent  processes  use  the  information  obtainable  from  the  syllabus 
as  an  input. 

For  instance,  sequencing  has  a  direct  relationship  with  task 
listing.  Sequencing  uses  the  elements  produced  during  job  and 
objectives  hierarchy  analysis  as  raw  materials.  Individual  tasks 
identified  during  job  analysis  become  the  backbone  elements  of 
the  syllabus,  and  individual  instructional  objectives  form  the 
prerequisite  events  leading  up  to  them.  In  some  cases,  interme¬ 
diate  level  instructional  events  not  derived  by  the  analyses  are 
inserted  during  sequencing,  but  these  are  comparatively  few,  and 
the  rules  for  adding  them  are  systematic.  From  this,  the  impor¬ 
tance  of  good  lob  and  task  analyses  can  be  seen.  Since  the  pro¬ 
ducts  of  anal  .is  translate  directly  into  syllabus  events,  the 
syllabus  is  limited  to  the  extent  that  the  analysis  is  inaccurate 
or  lacks  completeness  or  detail. 


Planning  for  performance  measurement  affects  the  syllabus. 
The  performance  measurement  plan  specifies  a  policy  concerning 
review  and  recertification  through  periodic  performance  checks. 
During  both  RTU  instruction  and  operational  assignments  this 
program  must  be  carried  on.  Reviews  and  recertification  proce¬ 
dures  must  be  built  into  the  syllabus  either  in  ways  which  take 
advantage  of  already  scheduled  events,  or  as  special  evaluation 
events.  During  syllabus  design  it  is  important  to  see  to  it  that 
all  required  review  and  recertification  needs  expressed  in  the 
measurement  plan  have  been  met. 

Target  population  studies  influence  syllabi  because  they 
define  the  entry-level  behavior  of  students.  Syllabi  are 
designed  to  ke  students  from  that  level,  so  it  is  natural  that 
a  careful  ider tif ication  of  entry  levels  can  assist  a  developer 
in  creating  a  syllabus  which  does  not  contain  unnecessary  events. 


Problem  analysis  and  constraint  analysis  both  constitute 
studies  of  the  training  environment,  its  practices,  and  its 
resources  and  have  important  effects  upon  syllabus  building 
because  they  define  the  restrictions  within  which  the  developer 
must  frame  the  syllabus  or  the  contraints  which  the  developer 
must  seek  to  change  to  accommodate  the  syllabus. 


Media  selection  and  sequencing  interact  intimately  with  each 
other  as  concurrent  processes,  par ct icular ly  in  the  area  in 
training  device  specification.  During  media  selection,  charac¬ 
teristics  of  training  devices  are  generated.  These  have  a  direct 
bearing  on  the  syllabus  which  is  generated  and  determine  types  of 
events  which  may  be  included  within  it.  In  turn,  once  media 
selections  have  been  made,  the  sequence  is  examined  carefully  to 
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insure  that  the  correct  progression  is  maintained  from  training 
device  to  training  device  and  that  local  overuse  or  overschedul¬ 
ing  of  one  training  device,  particularly  of  training  devices, 
does  not  occur  in  sequences  the  syllabus  call  for. 

Steps  following  sequencing  are  very  much  affected  by  the 
syllabus  produced  during  sequencing.  During  the  training  support 
requirements  analysis  (TSRA)  process  which  occurs  directly  after 
sequencing,  a  determination  of  the  total  cost  of  administering 
the  training  system  is  made.  Some  of  the  assumptions  of  this 
process  are  the  number  and  types  of  instructional  events  and 
levels  of  resource  utilization  called  for  by  the  syllabus. 

During  production  of  instructional  materials,  the  order  in 
which  materials  are  produced  is  sometimes  determined  by  the  order 
in  the  syllabus,  particulary  when  impending  deadlines  are 
evident.  At  any  rate,  the  list  of  instructional  events  in  the 
syllabus  is  the  sole  determiner  of  the  inventory  of  instructional 
materials  and  support  items  which  must  be  produced. 

During  implementation  of  the  instructional  system,  a 
syllabus,  which  is  cast  ultimately  in  the  form  of  a  day-to-day 
schedule  of  student  events,  serves  as  the  menu  for  student 
instructional  events  and  the  benchmark  against  which  student 
progress  through  training  is  measured. 


2.2  Principles  Utilized  in  Syllabus  Building 

Syllabus  building  is  an  engineering  process  which  attempts 
to  balance  the  best  available  principles  of  the  psychology  of 
learning  with  the  most  efficent  use  of  training  resources  and  the 
organization  of  the  learning  sequence  most  likely  to  provide 
motivation  for  the  syllabus  users  (both  students  and  instruc¬ 
tors)  .  It  is  possible  to  conceive  a  syllabus  carried  out 
entirely  within  the  aircraft  cockpit,  without  classrooms  or 
intermediate  training  device  use.  This  type  of  syllabus  is  a 
not-too-distant-past  reality  and  the  tendency  to  use  aircraft  as 
the  only  training  device  is  still  a  tendency  in  some  communities. 
Such  syllabi  are  highly  motivating  to  students  and  instructor 
uses  but  are  not  efficient  in  resource  use  and  most  often  are  not 
sound  in  terms  of  learning  principles.  Likewise,  syllabi 
designed  with  economic  factors  as  the  main  driving  force  are 
often  impoverished  in  their  attention  to  the  realities  of  learn¬ 
ing  and  user  motivation.  Though  these  syllabi  are  lower  in  imme¬ 
diate  cost,  there  is  reason  to  believe  that  long-range  perform¬ 
ance  and  motivation  outcomes  suffer  from  the  overemphasis  on  the 
economy. 

In  order  to  achieve  this  balance,  prinicples  from  each  of 
the  three  main  areas — psychological  principles,  economic  optimi¬ 
zation  principles,  and  user  motivation  pr inciples--must  be  con¬ 
sciously  applied. 
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Tables  1,  2,  and  3  list  several  of  the  more  important 
principles  in  each  area  and  the  implications  they  hold  for  the 
syllabus-building  procedure.  Various  features  of  the  sequencing 
procedure  described  in  this  report  are  intended  to  carry  out  the 
implications  in  actual  practice  and  to  insure  that  syllabi 
created  do  maximize  the  use  of  those  beneficial  principles  and 
avoid  inefficiency. 


2.3  Desirable  Syllabus  Characteristics 

Syllabi  should  possess  certain  characteristics  for  aircrew 
training  purposes.  Some  of  the  most  important  are  listed  below. 


2.3.1  Deliberateness,  Generative  Creation 

Syllabi  should  be  produced  through  a  systematic,  deliberate 
process,  and  the  logic  used  to  produce  them  should  be  available 
for  review.  That  is,  the  trail  of  decisions  generated  during 
production  of  the  syllabus  should  be  available  for  inspection  by 
those  responsible  for  maintaining  it.  Syllabi  produced  in  this 
fashion  will  be  revisable  as  instructional  content,  resources, 
best  knowledge  from  research  concerning  training  sequence,  and 
aircraft  mission  emphasis  change.  Since  the  syllabus  is  a  basic 
instructional  system  document  it  must  be  kept  current  and  will  be 
revised  periodically  as  conditions  dictate.  Being  able  to  change 
it  on  rational  and  logical,  rather  than  subjective  bases,  makes 
it  more  likely  that  the  syllabus  will  remain  an  efficient  one. 

One  implication  of  this  position  is  that  both  the  syllabus 
process  and  product  will  be  more  complex,  will  require  more 
manpower  to  maintain,  and  will  present  heavy  bookkeeping  and 
document  update  requirements  to  the  instructional  system.  The 
requirement  for  this,  though  it  is  a  discouraging  prospect  to 
system  operators,  is  analogous  to  a  similar  requirement  placed 
upon  materials  manufacturers  and  airframe  engineers  as  the 
materials  used  in  aircraft  were  made  to  increasingly  demanding 
specifications  and  required  more  and  more  complex  processes  to 
produce.  Once  the  commitment  was  made  to  the  added  data  manip¬ 
ulations  required,  it  was  found  that  it  enabled  more  sophisti¬ 
cated  applications  in  the  form  of  faster,  stronger,  lighter 
aircraft . 

Syllabi  generated  deliberately  will  be  more  difficult  to 
generate  and  maintain  but  are  necessary  if  instructional  systems 
are  to  increase  the  precision  and  efficiency  of  effect  during 
operation . 


2.3.2  Cost  Minimization 

Since  maximum  training  efficiency  levels  are  one  of  the  main 
concerns  of  the  syllabus,  minimizing  costs  where  possible  without 
degrading  training  must  be  one  of  its  characteristics.  The 
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Table  1 — Psychological  principles  related  to  syllabus 

development . 


Title 


Implication  for 

Principle  Syllabus  Building 


Information 

overload 

Students  are  able  to 
intake  only  a  limited 
amount  of  new  factual 
informatin  in  a  given 
time  span. 

Space  factual  learning 
appropriately  to  avoid 
information  overload. 

Speed  of 
forgetting 

Forgetting  begins  to 
occur  almost  immedi¬ 
ately  unless  testing  or 
use  of  the  information 
occurs  to  fix  informa¬ 
tion  in  memory. 

Test  soon  after  learn¬ 
ing  of  factual  informa¬ 
tion. 

Provide  opportunities 
for  use  of  factual 
information  in  perform¬ 
ance  as  soon  as  possi¬ 
ble  after  learning. 

Drill  and  test  period¬ 
ically  for  review  and 
refresher  purposes. 

Information 

integration 

requirement 

Integration  of  new 
information  into  exist¬ 
ing  knowledge  struc¬ 
tures  requires  relating 
it  to  previously 
learned  information. 

Prescribe  regular 
reviews  and  cumulative 
syllabus  structure 
incorporating  old  and 
new  learning  into 
longer  sequences. 

Minimum  motor 
skill  inte¬ 
gration  time 

Integration  of  motor 
skills  into  longer  com¬ 
plexes  of  motor  skills 
requires  a  certain  min¬ 
imum  amount  of  practice 
to  achieve  smooth,  con¬ 
sistent  behaviors  which 
will  resist  loss. 

Generation  of  syllabus 
based  on  best  estimated 
required  training  time 
rather  than  on  the 
basis  of  resource  con¬ 
straints. 

Interference 

The  learning  of  some 
tasks  and  information 
interfere  with  the 
learning  of  other  tasks 
and  information  when 
confusing  similarities 
exist  between  them. 

Separate  interfering 
tasks  in  the  syllabus 
sequence  or  provide 
adequate  emphasis  and 
discriminative  over¬ 
practice  if  they  are  to 
be  taught. 

Table  1  (cont.) 


Title 


Transf er 


Mastery  of 
basics 


Pr inciple 


Learning  of  some  skills 
faciliates  the  learning 
of  certain  other  skills 
because  of  similarity 
in  task  sequence,  simi¬ 
larity  of  component 
subtasks,  similarity  of 
cognitive  processing 
demands,  or  similarity 
of  knowledge  required. 

Basic  building  blocks 
of  higher  level  behav¬ 
iors  should  be  well- 
learned  to  make  combin¬ 
ing  into  longer  chains 
of  behavior  easier, 
less  demanding,  and 
less  wastefull  of 
resources  through 
student  error. 


Implication  for 
Syllabus  Building 

Put  high  transfer  tasks 
in  sequence  such  that 
learning  of  one  task 
may  be  allowed  to  bene¬ 
fit  learning  of  subse¬ 
quent  tasks. 


Insure  that  basics  are 
practiced  sufficiently 
and  avoid  the  tempta¬ 
tion  to  move  quickly  to 
complex  behaviors  when 
early  behaviors  appear 
to  be  easily  acquired. 
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Table  2 — Economic/optimization  principles  related  to 
syllabus  development. 


Title  Principle 


Continous  The  syllabus  sequence 

flow  must  not  schedule 

training  devices  in 
such  a  way  that  demand 
for  a  given  training 
resource  experiences 
peaks  and  valleys  which 
create  bottlenecks  and 
cause  students  to  wait 
for  the  use  of  the 
resource . 

Least  cost  The  syllabus  must 

provide  a  least  total 
cost  method  of  training 
students  to  a  given 
level  of  proficiency 
through  the  use  of  a 
mixture  of  high-  and 
low-cost  training 
resources . 


’’  I 

* 

''  J 
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Implication 


Spread  sequence  events 
as  much  as  possible 
over  a  range  of  train¬ 
ing  devices,  avoiding 
over  dependence  upon 
one  or  a  small  number 
of  devices,  particu¬ 
larly  the  ones  in  short 
supply  or  which  are 
expensive  to  acquire. 


Select  sequences  which 
make  maximum  use  of 
lower  cost  training 
devices  and  bring  stu¬ 
dents  to  higher  cost 
devices  only  after 
adequate  preparation  in 
basic  skills  and  know¬ 
ledge. 
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Table  3 — User  motivation  principles  related  to 
syllabus  development. 


Title 


Pr inciple 


Implication 


A  "now"  prog-  Student  and  instructors 
ress  to  be  should  be  able  to  per- 

perceivable  ceive  that  progress 

toward  long-  and  short- 
range  course  goals  is 
being  made  during  each 
study  and  performance 
event . 


Maximize 

rewards 

available 


Allow  students  to  earn 
rewarding  opportunities 
through  excellence  in 
study  and  performance. 


Avoid 

punishing 

contingencies 


High  achievement  should 
not  be  rewarded  with 
punishing  experiences, 
which  include  the  with¬ 
drawal  of  opportuni¬ 
ties. 


Provide  early  hands  on 
practice  as  soon  as 
possible  with  real 
equipment  within  limits 
of  safety. 

Avoid  long  sequences  of 
study  or  practice  which 
show  little  apparent 
payoff  in  skills  or 
relevant  knowledge. 

Build  syllabus  study 
units  in  terms  of  time 
required  to  completion 
as  much  as  possible. 

Design  syllabus  events 
to  allow  accelerated 
students  to  meet  chal¬ 
lenges  slightly  above 
those  encountered  by 
average  students. 

Do  not  eliminate 
opportunities  for 
additional  flying  or 
study  on  the  basis  of 
early  success. 


problems  of  complexity,  manpower,  and  bookkeeping  requirements 
described  in  the  previous  paragraph  are  minimized  substantially 
when  compared  to  the  potential  they  create  for  savings.  When 
syllabi  are  written  for  expensive  (high  volume  or  high  cost) 
training  systems  like  those  found  in  aircrew  training,  small 
changes  in  the  syllabus  can  effect  large  savings  in  the  cost  of 
training.  How-wer,  most  of  these  small  changes  are  not  possible 
without  a  flexible  process  made  possible  by  the  additional 
recordkeeping  and  personnel  already  mentioned. 


2.3.3  Empirically  Based 

During  the  generation  of  a  syllabus  certain  assumptions  are 
made  concerning  the  amount  of  instruction  and  practice  required 
by  a  student  at  a  given  entry  level.  As  the  syllabus  is  imple¬ 
mented  it  can  be  empirically  determined  from  students'  actual 
performance  data  how  accurate  those  assumptions  were,  and  as  this 
empirical  data  base  grows,  the  syllabus  can  be  changed  to  adjust 
incorrect  assumptions.  Until  a  syllabus  has  been  through  this 
validation  process  it  should  be  treated  as  a  tentative  document. 

One  requirement  to  make  possible  the  empirical  data  feedback 
described  here  is  a  precise  and  reliable  performance  measurement 
system  and  the  commitment  on  the  part  of  the  training  management 
and  instructor  to  observe  the  standards  of  the  performance 
measurement  system  for  progressing  and  graduating  students. 


2.3.4  Flexibility  to  Individual  Student  Requirements 

If  the  generative  syllabus  technology  presented  in  this 
paper  were  carried  out  to  its  ultimate  extent,  a  syllabus  tai¬ 
lored  specifically  to  individual  students  and  based  on  each  one's 
own  learning  history  could  be  created.  Such  a  practice  is 
probably  not  far  off  in  the  future  if  the  techniques  of  syllabus 
generation  improve.  It  would  be  desirable  from  a  learning  and 
performance  point  of  view  as  well  as  an  economic  one.  Since  all 
students  do  not  learn  at  the  same  pace,  however,  and  since 
individual  syllabi  are  not  possible,  a  syllabus  must  be  construc¬ 
ted  for  a  particular  level  of  student  (the  fast  student,  the  slow 
student,  or  the  average  student),  and  a  major  decision  must  be 
made  concerning  the  flexibility  which  will  be  allowed  in  the 
syllabus  to  accommodate  students  which  differ  from  the  type.  If 
the  syllabus  is  to  accommodate  the  needs  of  slower  students, 
additional  instructional  events  must  be  available  so  slower  stu¬ 
dents  may  have  the  necessary  level  of  practice.  Faster  students 
progress  ahead  of  others,  and  either  the  option  of  early  gradua¬ 
tion  from  the  course  or  the  opportunity  for  advanced  instruc¬ 
tional  events  during  the  course  must  be  provided  to  maintain  the 
motivation  of  those  students  and  to  make  the  best  use  of  their 
unique  talents.  (See  project  report  no.  14,  "Recommendations  for 
the  F-16  Performance  Measurement  System"  for  an  example  of  this.) 
The  syllabus  must  be  flexible  enough  to  prescribe  instructional 
events  for  all  classes  of  students. 
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3.0  SYLLABUS  STRUCTURE,  CRITERIA,  AND  MASTERY  MODELS 


The  development  of  a  syllabus  for  aircrew  training  must  be 
closely  related  to  the  principle  of  criterion-referenced  instruc¬ 
tion  and  testing.  This  section  presents  a  discussion  of  the 
relation  between  syllabus  structure  and  criterion-referencing. 


3.1  Syllabus  Structure  for  Aircrew  Training 

Figure  2  is  a  diagramatic  representation  of  the  structure  of 
syllabus  for  aircrew  training  and  for  competency  training  in 
general  as  well.  Main  parts  of  the  syllabus  are:  (1)  terminal 
course  evaluation,  (2)  intermediate  evaluation  points  at  the  end 
of  phases  or  sequences  of  training,  (3)  sequences  of  real-world 
or  simulated  real-world  practice  of  developing  behaviors,  (simu¬ 
lator  and  aircraft  practice) ,  (4'  an  intermediate  level  of  simu¬ 
lated  exercises  of  highly  fragmented  performances  related 
directly  to  the  real-world  exercises  but  much  more  limited  in 
scope  (isolated  task  pract;ce  in  non-job  environments),  and  (5) 
series  of  academic  instru^.ion  Prerequisites ,  providing  the 
student  with  the  verbal  and  cognitive  skills  prerequisite  to 
attempting  performance  of  ;  a.-ks.  Note  that  the  academics/iso¬ 
lated  task  practice/real-woL  id  task  practice  sequence  is  cycled 
through  repeatedly.  Note  also  that  several  academic  prerequi¬ 
sites  may  precede  individ'ial  behavior  practice  and  that  several 
individual  behaviors  are  prerequisite  to  job  behavior  practice. 

All  instructional  events  contained  in  the  syllabus  come  from 
three  sources:  (1)  instructual  events  derived  directly  from  the 
job  task  listing,  the  criterion-referenced  objectives,  and  the 
objectives  hierarchy  (the  major  source)  ,  (2)  syllabus  events 
created  to  approximate  job  behavior  which  are  either  too  danger¬ 
ous,  too  expensive,  or  too  difficult  for  students  to  attempt 
during  early  stages  of  training,  and  (3)  instructional  events 
allowing  the  student  to  review  and  combine  previously-learned 
behaviors  or  attempt  them  in  more  complex  performance  environ¬ 
ments  . 


3.2  Criterion-referenced  Syllabi 

Criterion-referenced  syllabi  assume  that  the  goal  of  a 
course  of  training  is  a  performance  level  which  students  are 
expected  to  reach.  It  is  an  axiom  of  criterion-referencing  that 
students  do  not  graduate  until  they  have  reached  the  course 
performance  goal  at  the  expected  level  of  precision.  The  course 
performance  goal  is  sometimes  called  a  "mastery  model",  the  model 
of  the  behavior  students  are  to  master  in  the  course.  It  con¬ 
sists  of  a  statement  of  behaviors  students  are  expected  to  master 
and  the  quality  and  precision  standards  associated  with  each. 

For  F-16  behaviors,  conditions  and  performence  standards  are 
expressed  in  criterion-referenced  objectives.  (See  project 
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Figure  2  -  Diagrammatic  Representation  of  Syllabus  Structure 
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report  no.  5,  "Derivation,  Formatting,  and  Use  of  Criterion- 
referenced  Objectives  (CROs)  and  Criterion-referenced  Tests 
(CRTs).")  In  Figure  2,  the  terminal  course  evaluation  is  a 
measurement  of  student  attainment  of  the  mastery  model.  The 
problems  encountered  in  making  criterion  statements  of  this  type 
are  discussed  in  project  report  no.  14,  "Recommendations  for  the 
F-16  Performance  Measurement  System". 

Not  only  are  terminal  course  mastery  checkpoints  required 
for  criterion-referenced  syllabi,  but  intermediate  checkpoints  as 
well.  Figure  2  represents  formal  intermediate  evaluation  points 
for  which  aircrew  trainin,  might  represent  end-of-phase  evalua¬ 
tions.  Just  as  students  do  not  graduate  from  a  course  until 
performance  has  reached  criterion  (mastery  model)  level,  they 
should  not  pass  by  these  intermediate  evaluation  points  until 
they  have  met  the  applicable  performance  standards.  Remedial 
instructions  and  retesting  may  be  necessary  for  students  to 
accomplish  this,  and  within  resource  expenditure  limits  some 
students  may  find  it  impossible. 

The  architecture  of  evaluation  points  described  does  not 
exhaust  the  possible  use  of  criterion-referencing  in  syllabus 
structure.  The  exit  from  virtually  every  main  block  of  instruc¬ 
tional  experience  may  be  used  as  a  checkpoint  of  the  effective¬ 
ness  of  foregoing  instruction  and  the  students'  readiness  for 
subsequent  experiences.  Consideration  in  naming  the  checkpoints 
actually  selected  must  beware  of  overtesting  yet  insure  that 
students  are  well-prepared  to  make  the  best  use  of  more  expensive 
training  media  involved  in  higher  levels  of  task  practice. 

The  F-16  syllabus  will  be  a  criterion-referenced  syllabus. 
Without  such  a  syllabus  it  is  difficult  to  conduct  the  training 
of  pilots  within  any  secure  knowledge  of  the  capabilities  of 
course  graduates.  The  introduction  of  criterion-referenced 
structure  into  the  F-16  syllabi  makes  possible  not  only  a  careful 
recording  of  student  attainment  of  intermediate  and  terminal 
course  goals,  but  careful  recording  of  student  progress  toward 
those  goals,  a  diagnosis  of  areas  of  student  performance  defi¬ 
ciency  (collective  and  individual) ,  and  the  ability  to  insure 
that  training  equipment  and  resources  will  be  used  only  by 
students  who  are  ready  to  obtain  the  maximum  benefit  from  their 
use.  In  the  long  run  this  means  more  efficient  use  of  training 
resources . 


3.3  Criterion-referencing  and  Career  Performance  Goals 

Only  very  seldom  is  a  course  of  training  an  end  in  itself. 

A  given  course  is  very  often  but  a  single  phase  of  a  career  or 
long-term  training  plan.  Because  the  various  portions  of  a 
career  training  progression  must  fit  together,  complement  each 
other,  and  act  as  a  correlated  sequence,  it  is  proper  to  look 
upon  the  syllabus  for  each  of  them  as  a  part  of  a  larger,  contin¬ 
uous  training  course.  It  is  desirable  that  this  training  course 
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have  a  syllabus  (a  composite  of  the  individual  course  syllabi) 
which  is  criterion-referenced,  including  a  course  mastery  model. 
The  course  mastery  model  in  this  case  is  a  career  mastery  model 
as  well,  and  the  individual  course  terminal  evaluation  points 
become  intermediate  evaluation  points  to  the  career  syllabus 
(using  the  Figure  2  terminology  and  structure) . 

For  pilot  training  in  general  some  manifestations  of  a 
criterion-referenced  career  syllabus  structure  are  apparent  in 
the  progression  of  pilots  from  basic  training  through  career 
mastery  attainment.  A  student  technically  does  not  progress  from 
school  to  school  (i.e.,  basic  EFLIT,  RTU,  Operational  Squadron) 
unless  he  meets  certain  minimum  performance  criteria.  However, 
whether  the  career  syllabus  constructed  in  this  way  is  actually 
functioning  as  a  criterion-referenced  mechanism  depends  upon  the 
adequacy  with  which  course  mastery  models  and  terminal  behaviors 
have  been  specified  and  the  degree  to  which  testing  is  truly 
criterion-referenced  rather  than  subjective- judgment-based . 

This  perspective  of  individual  course  syllabi  fitting  within 
the  career  course  syllabus  to  form  a  larger  criterion-referenced 
syllabus  structure  is  a  very  important  perspective.  Different 
stages  (courses)  in  a  pilot's  training  often  do  not  require  the 
training  of  all  new  behaviors  but  instead  require  the  raising  of 
many  behaviors  already  acquired  at  a  basic  level  of  competence  to 
a  higher  performance  standard  or  to  capability  for  performance 
under  increasingly  difficult  circumstances. 

The  designation  of  levels  of  intermediate  criterion  perform¬ 
ance  within  a  course  (as  well  as  between  courses)  is  desirable 
over  designation  of  a  single  terminal  performance  level  for 
motivational  and  administrative  purposes.  Motivationally  it  is 
important  for  the  student  to  perceive  his  own  progress  in  attain¬ 
ing  a  body  of  skills  and  becoming  qualified  as  a  pilot.  In  the 
administration  of  aircrew  training,  goals  must  be  stated  for 
student  performance  which  can  be  achieved  within  a  reasonable 
training  period  so  that  it  can  be  determined  whether  student 
progress  is  satisfactory  and  if  appropriate  amounts  of  training 
resources  are  being  spent  for  particular  students.  Moreover, 
between-syllabus  levels  of  performance  become  important  because 
they  define  to  commanders  the  real  performance  capabilities  of 
the  pilots  under  their  command,  thus  determining  the  ways  in 
which  these  pilots  may  be  used  to  complete  missions.  A  student 
will,  in  the  course  of  his  flying  career,  reach  very  high  levels 
of  precision  in  control  of  an  aircraft,  but  only  a  certain  mini¬ 
mum  level  of  control  is  necessary  before  he  is  ready  to  progress 
to  more  involved  behaviors  requiring  control  to  be  exercised. 
Control  continues  to  be  developed  in  this  fashion  as  the  student 
progresses  to  more  sophisticated  behaviors.  It  is  important  to 
emphasize  that  this  is  not  a  recommendation  that  the  stuuent  be 
advanced  to  more  complex  behaviors  before  he  has  established  a 
solid  basic  competence,  but  it  is  a  recommendation  that  once 
minimum  competence  has  been  reached  training  proceed  into  new 
areas. 
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4.0  MODEL  OF  A  DEVELOPING  SKILL 


To  make  possible  a  systematic,  generative  syllabus  develop¬ 
ment  procedure,  a  model  has  been  constructed  to  describe  the 
process  training  of  a  single  skill.  It  contains:  (1)  optional 
instructional  events  required  during  training,  (2)  their 
sequence,  and  (3)  the  instructional  variables  which  may  be  manip¬ 
ulated  during  that  sequence.  This  model,  which  is  diagrammati- 
cally  presented  in  Figure  3  and  is  described  in  this  section,  is 
used  as  the  basic  building  block  of  the  syllabus.  It  makes  the 
following  assumptions: 

1.  Student  ability  to  perform  tasks  requires  instruction 
which  can  most  efficiently  be  administered  outside  of 
training  device  and  aircraft.  Though  it  may  be  desir¬ 
able  to  review  that  instruction  or  use  media  aids  as 
adjuncts  to  the  use  of  training  devices,  it  is  assumed 
that  training  device  time  can  best  be  economized  by 
having  the  student  ready  to  use  the  device  by  appropri¬ 
ate  prior  instruction — not  only  in  procedures  to  be 
executed,  but  in  the  use  of  the  device  itself. 

2.  A  general  sequence  of  training  experiences  drawing  the 
student  from  more  simple  to  more  complex  performance 
environments  is  optimal  in  terms  of  developing  skills 
with  least  resource  consumption. 

3.  A  procedure  of  training  individual,  isolated  skills  to 
certain  minimum  tested  levels  of  proficiency  is  in  the 
long  run  more  economical  than  moving  to  more  complex 
combinations  of  skill  too  soon.  Students  are  expected 
to  derive  the  maximum  benefit  from  training  resource 
utilization  only  if  they  are  fully  ready  for  the 
instructional  events  they  encounter. 

The  skill  developing  model  is  presented  in  Figures  3  and  4 
in  two  parts.  Figure  3  (which  is  a  more  detailed  version  of 
Figure  2)  presents  the  full  context  of  instructional  events  in 
skill  development,  and  Figure  4  presents  specific  detail  on  the 
use  of  training  devices  within  that  context.  As  anticipated  by 
the  assumptions  listed  above  and  as  shown  in  Figure  4,  academic 
training  forms  a  first  phase  of  skill  development  with:  (1)  the 
statement  of  verbal  information  necessary  for  skill  execution, 

(2)  adequate  drill  and  practice  on  difficult  memorized  informa¬ 
tion,  (3)  practice  in  difficult  discriminations  (classifications) 
and  rule-applications,  (4)  individual  and  group  solving  of  job¬ 
like  problems,  (5)  tie-together  where  necessary  of  interrelated 
information  items  into  a  coherent  background  knowledge-base,  and 
(6)  non-equipment  related  simulated  cognitive  practice  and 
rehearsal . 

It  is  a  critical  feature  of  the  model  that  academic  training 
related  to  the  performance  of  a  particular  skill  be  placed  imme- 
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Highest  Level 


Figure  4  -  Training  Events  in  Skid  Development  Affecting  Syllabus  Dimensions 


diately  prior  to  the  time  when  students  will  begin  practicing  the 
skill  or  its  subskills.  The  diagram  is  intended  to  suggest  that 
several  academic  training  segments  may  relate  to  subskill  train¬ 
ing  and  that  practice  in  several  isolated  subskills  may  be  pre¬ 
requisites  to  more  advanced  training.  By  insuring  that  low-level 
skills  are  well-trained  in  this  way,  it  is  expected  that  the 
training  of  higher  level  skills  can  be  executed  using  fewer 
resources.  This  in  fact  is  only  an  extension  of  the  philosophy 
behind  simulator  use  to  its  logical  conclusion. 

A  second  stage  of  skill  development  interwoven  in  sequence 
appropriately  with  the  academic  training  is  the  practice  of 
particularly  difficult  or  critical  subskills  isolated  from  their 
present  skills.  The  media  selection  process  described  in  project 
report  no.  20,  "F-16  Media  Selection  and  Utilization  Plan 
Report,"  contains  a  series  of  steps  (see  Section  3.0  in  project 
report  no.  20)  whose  purpose  it  is  to  identify  any  of  those 
critical,  often  subtle  skills  which  may  have  escaped  the  analysts 
during  task  analysis.  Often  these  skills,  which  may  seem  unim¬ 
portant,  can  hinder  a  student's  performance  under  high  stress  or 
precision  timing  conditions  if  they  are  not  adequately  pre¬ 
trained.  These  skills  may  be  practiced  most  often  on  relatively 
simple  two  and  three  dimensional  simulation  equipment  with  low- 
fidelity  and  interaction  requirements. 

The  third  stage  in  skill  development  is  the  practice  parent 
skills  and  combinations  of  skills.  These  sequences  represent 
near- job-1 ike  performance  requirements  in  near- job-1 ike  environ¬ 
ments.  Though  the  total  job  environment  need  not  be  present 
during  performance,  the  most  important  features  relating  to  task 
performance  must  be.  A  high  level  of  fidelity  as  far  as  task 
stimuli,  response  capabilities  and  requirements  for  timing,  and 
blind  performance  rust  be  incorporated  into  the  training. 

The  final  step  in  skill  develo/  lent  is  the  exercise  of  the 
skill  in  real-world  job  environments  or  high-fidelity  simulation 
of  them.  Practice  of  skills  in  this  environment  should  cover  the 
entire  spectrum  of  operational  conditions  and  the  full  range  of 
job  demands  combined  together  into  complex  scenarios. 

Two  principles  which  apply  to  this  sequenced  progression 
through  training  devices  are: 

1.  Once  basic  simulation  levels  have  been  passed  and 

students  have  encountered  more  sophisticated  performance 
environments,  requirement  for  the  student  to  regress  to 
the  simpler  environments  for  further  practice  should  be 
questioned  unless  the  student  is  encountering  difficulty 
in  the  more  complex  environment.  It  is  only  recommended 
by  this  report  to  the  extent  that  demonstrated  validity 
of  simulator  practice  can  be  made  for  F-16  tasks  with 
F-16  training  devices. 
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2.  Students  may  make  considerable  progress  between  prac¬ 
tices,  but  caution  should  be  exercised  against  moving 
the  student  off  basic  exercises  before  he  begins  to  feel 
confident.  Caution  against  too  fast  an  advancement 
through  the  syllabus  is  also  given.  It  is  possible  that 
students  showing  high  capability  levels  early  can  be 
later  penalized  if  they  find  their  mastery  of  basic 
tasks  weak. 

Figure  4  describes  the  training  events  which  may  take  place 
in  the  succession  of  traing  devices  described  in  Figure  3. 

Figure  4  shows  instructional  events  in  the  learning  of  an 
individual  task  for  one  training  device. 

Focusing  specifically  on  the  development  of  a  skill  in  the 
highest  level  training  device,  the  aircraft  in  this  case,  the 
first  step  in  skill  development  is  a  preassessment  (P)  of  the 
student's  ability  to  perform  the  task  at  the  criterion  level.  If 
the  student  can  accomplish  that,  there  is  no  need  to  proceed  with 
training,  and  the  student  should  progress.  If  the  student  cannot 
perform  to  criterion,  training  in  that  device  should  proceed. 
(NOTE:  The  evaluation  taking  place  in  preassessment  may  be 

highly  informal  if  the  student  and  instructor  are  sure  the 
student  is  not  able  to  perform  the  task.  Preassessment  may  take 
the  form  of  a  verbal  exchange  in  such  cases.  If  the  student 
feels  capable,  however,  and  there  are  no  safety  risks  to  prevent 
him  from  attempting  the  task,  a  more  formal  performance  measure¬ 
ment  procedure  should  take  place.  In  aircrew  training  syllabi, 
where  transition  of  experienced  pilots  from  one  aircraft  to 
another  is  being  made,  this  preassessment  of  the  incoming  pilots' 
already-mastered  skills  can  be  a  contributor  to  the  efficient 
operation  of  the  instructional  system. 

Following  preassessment  should  come  an  appropriate  form  of 
demonstration  (D) .  This  may  include  single  or  multiple  demon¬ 
strations,  and  the  student  may  view  it  (them)  from  different 
perspectives — as  a  passenger  in  a  craft  piloted  by  the  demon¬ 
strator,  or  from  another  craft.  Moreover,  demonstrations  may  be 
live  or  vicarious,  such  as  on  video  or  even  audio  tape.  Demon¬ 
strations  may  vary  according  to:  (1)  level  of  realism,  (2)  the 
timing  of  the  demonstration,  (3)  the  observability  by  the  student 
of  relevant  acts  which  are  a  part  of  the  performance  of  the 
demonstrated  skill,  (4)  the  complexity  and  length  of  the 
demonstrated  task,  or  (5)  in  the  number  of  repeated  demonstra¬ 
tions  which  the  student  is  allowed  to  observe. 

Following  the  demonstration  it  is  desirable  for  the  student 
to  practice  (P)  the  skill  with  whatever  cues,  prompts,  and  feed¬ 
back  necessary.  During  the  course  of  practice  these  should  be 
removed  systematically,  with  the  exception  of  feedback,  which  is 
always  necessary.  Dimensions  relative  to  practice  are  (1)  real¬ 
ism  of  the  performance  environment,  (2)  number  of  grouped  or 
massed  practices  allowed  within  a  closed  time  frame,  (3)  number 
of  spaced  practices  (the  number  of  different  practice  sessions) , 
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(4)  the  elapsed  time  between  practices,  and  (5)  the  level  of 
cueing,  prompting,  and  feedback  which  the  student  receives  during 
each  performance  of  a  skill.  It  is  assumed  that  informal 
performance  measurement  takes  place  every  time  the  student 
performs  the  skill  during  practice.  For  administrative  and 
student  motivational  purposes  and  progress  reporting,  formal 
measurement  of  the  best  performance  during  a  practice  session 
should  also  be  made.  The  results  of  that  measurement  serve  as  a 
benchmark  of  student  progress  and  training  effectiveness. 

Following  practice,  when  it  has  been  determined  that  the 
student  is  ready  to  challenge  the  criterion,  formal  evaluation 
(E)  may  take  place.  The  dimensions  of  this  evaluation  include 
(1)  the  level  of  cueing  and  prompting  allowed  by  instructors,  (2) 
the  number  of  attempts  allowed,  (3)  number  of  spaced  evaluations 
required  for  certification,  and  (4)  the  elapsed  time  between 
evaluations . 

Performance  assessment  policies  are  discussed  in  project 
report  no.  14,  "Recommendations  for  the  F-16  Performance  Measure¬ 
ment  System."  The  performance  measurement  system  discussed  there 
with  its  detailed  record  of  student  performance  and  the  teaching 
made  possible  for  specific  areas  of  student  growth,  makes  a 
system  of  continuous  evaluation  possible.  Normally  it  would  be 
recommended  that  the  student  be  evaluated  multiple  times  to 
insure  that  good  scores  were  not  the  result  of  a  fluke  or  an 
unusually  good  day  and  that  grades  recorded  represent  real  and 
consistent  behavior  levels.  The  opposite  reasoning  applies,  as 
multiple  measurement  guards  the  student  against  bad  days  as  well. 
Having  a  performance  measurement  system  that  continually  monitors 
student  progress  allows  one-time  formal  evaluations  to  take  place 
backed  up  by  evaluations  made  during  practice  and  gives  the  eval¬ 
uator  some  estimate  of  the  consistency  and  stability  of  the 
students'  performace. 

Syllabi  are  concerned  with  both  the  establishment  and  the 
maintenance  of  skills,  and  one  of  the  dimensions  of  the  syllabus 
must  be  provision  for  periodic  practice  and  evaluation  of  skills 
already  mastered  in  such  a  way  that  they  may  be  systematically 
maintained.  Once  it  has  been  ascertained  that  a  student  has  met 
the  measurement  criteria,  a  periodic  re-evaluation  and  recertifi¬ 
cation  (R)  must  be  conducted  as  well  as  periodic  review  practice 
to  insure  adequate  maintenance  of  the  skill.  The  dimensions  of 
this  review  and  evaluation  function  are  much  the  same  as  for 
practice.  They  include:  (1)  The  level  of  realism  of  the 
performance  environment,  (2)  the  number  of  massed  practices,  and 
(3)  the  number  of  spaced  practices,  (4)  the  elapsed  time  between 
the  practices.  Cognitive  refresher  training  will  also  be  neces¬ 
sary  prior  to  these  review  events. 

A  note  of  caution  should  be  sounded  relative  to  refresher 
training  and  recertification.  The  specification  of  numbers  of 
practices  and  elapsed  time  between  practices  should  not  be 
interpreted  to  mean  that  those  are  the  criteria  for  recertifi- 


cation.  Flying  or  performing  in  a  simulator  a  certain  number  of 
certain  tasks  does  not  constitute  recertification  in  a  criterion- 
referenced  syllabus.  As  practices  are  concluded  during  the 
review,  formal  evaluation  should  take  place  using  criteria  speci¬ 
fied  at  the  students'  advanced  level  of  performance.  It  is  this 
measurement  which  consitutes  the  recertification.  It  is  mislead¬ 
ing  and  dangerous  to  say  that  because  a  student  has  survived  a 
certain  number  of  instructional  events,  that  he  is  qualified  in 
those  events.  Criterion  judging  prevents  that  from  happening. 

In  Section  5.0  the  description  of  the  the  sequencing 
procedure  will  show  that  the  general  model  of  skill  development 
is  the  fundamental  building  block  of  the  syllabus.  Using  this 
model  and  giving  specific  values  to  each  of  the  dimensions  for 
specific  tasks  structures  the  syllabus  from  within,  rather  than 
imposing  form  and  proportion  on  the  syllabus  through  resource 
constraints  or  preconcieved  ideas  of  the  developer. 

The  dimensions  assigned  to  this  structure  taken  cumulatively 
are  the  dimensions  of  the  syllabus,  for  the  syllabus  is  made  up 
of  a  collection  of  individual  task  training  structures  linked 
together  in  their  prerequisite  (as  determined  by  the  task  list¬ 
ing)  sequence  or  the  sequence  of  best  transfer  from  one  skill  to 
another.  This  is  demonstrated  in  the  inset  on  Figure  4  which 
shows  first  the  mastery  of  prerequisite  (or  high  transfer)  skills 
1  and  2  to  a  given  performance  criterion  and  then  the  practice  of 
skill  3  in  first  simple  and  then  complex  environments. 
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5.0  F-16  SYLLABUS  GENERATION  PROCEDURES 


This  section  contains  a  description  of  the  procedures  us^d 
in  the  generation  of  the  F-16  Syllabus.  Following  the  brief 
overview  a  step  by  step  review  of  the  procedures  is  presented. 


5.1  Generative  Syllabus  Construction  and  Maintenance 

The  principles,  models,  and  procedures  described  in  this  and 
the  preceding  sections  of  this  report  provide  a  practical  mechan¬ 
ism  which  can  be  used  to  generate  instructional  syllabi.  It 
should  be  apparent  to  the  reader  that  generating  syllabi  in  this 
fashion  rather  than  by  traditional  methods  produces  side  effects, 
including  an  increased  bookkeeping  load,  and  the  requirement  for 
subject-matter  expert  involvement  at  a  detailed  level  of 
decision-making  on  a  continued  basis.  A  procedure  which  seeks  to 
generate  a  syllabus  in  a  logical  way  from  data  must  expect  these 
results . 

Along  with  these  negative  side  effects,  the  syllabus  gener¬ 
ated  can  also  be  expected  to  have  some  useful  properties.  One 
anticipated  benefit  is  the  possibility  of  a  comparatively  precise 
self-adjusting  capability.  During  the  construction  of  a  syllabus 
using  the  generative  procedures  described  in  this  report,  best 
estimates  are  used  to  determine  the  dimensions  of  the  demonstra¬ 
tion,  practice,  evaluation,  and  review  events  for  each  task 
because  no  current  body  of  data  has  been  amassed  specifying 
values  for  those  variables.  A  well-managed  evaluation  and 
performance  measurement  system  can  produce  data  which  indicate 
the  adequacy  of  the  syllabus.  As  these  data  are  gathered  on 
student  and  syllabus  performance,  it  will  become  apparent  which 
dimensions  were  mis-est imated ,  and  corrections  to  the  syllabus 
may  be  made.  Over  time  this  will  make  possible  increased  effi¬ 
ciency  in  the  syllabus.  As  the  backgrounds  and  capabilities  of 
student  groups  entering  training  change,  the  syllabus  can  be 
systematically  adjusted  to  accommodate  fluctuation.  This  adjust¬ 
ment  will  be  possible  on  the  basis  of  operational  data  and  in  a 
way  not  possible  with  traditionally-constructed  syllabi.  Once 
this  procedure  is  adopted  and  syllabi  are  constructed  after  this 
fashion  and  feedback  loops  are  established  and  utilized,  data 
will  begin  to  accumulate  concerning  the  speed  with  which  given 
students  of  given  characteristics  can  be  trained  to  given  levels 
of  performance.  This  data  base  will  be  useful  not  only  in  the 
maintanence  of  existing  syllabi  but  also  in  the  generation  of  new 
syllabi  for  the  F-16  and  other  aircraft. 

The  alternative  is  the  creation  of  syllabi  through  methods 
which  are  not  generative  and  produce  syllabi  which  are  less 
efficient  and  less  responsive  to  data  from  actual  use.  It  is 
important  to  understand  that  a  syllabus  generated  by  this  method 
must  be  revised  by  this  method,  therefore  the  costs  of  generating 
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the  syllabus  initially  do  not  disappear  once  the  syllabus  has 
been  generated.  Care  must  be  taken  to  see  that  the  syllabus  is 
kept  in  an  updated  condition  continually.  In  a  cost-conscious 
environment  this  does  not  appear  on  the  surface  to  be  good  news, 
but  if  instructional  systems  are  to  become  more  efficient  in  the 
utilization  of  extremely  costly  training  resources  this  is  a 
commitment  which  trainers  must  make,  and  there  is  a  high  likeli¬ 
hood  that  it  is  an  activity  which  will  more  than  pay  for  itself 
in  the  long  run  through  savings. 


5.2  Overview  of  Syllabus  Generation  Procedures 

Syllabus  generation  as  described  in  this  report  can  be 
looked  upon  as  a  four-stage  process  consisting  of  (1)  definition 
of  mastery  models  and  terminal  performance  tasks,  (2)  determina¬ 
tion  of  specific  tasks  to  be  learned  and  the  levels  of  desired 
performance  for  each,  (3)  determination  of  the  syllabus  structure 
based  upon  using  the  aircraft  as  the  only  training  device,  and 
(4)  determination  of  possible  utilization  of  non-aircraft  train¬ 
ing  devices  to  relieve  aircraft  consumption. 


5.3  Step-by-Step  Sequencing  Procedure  Description 

This  section  contains  a  step-by-step,  description  of  the 
sequencing  process.  Figure  5  is  a  flow  chart  representation  of 
the  steps  in  the  process.  Each  is  discussed  individually  below. 


5.3.1  Step  Is  Define  the  Mastery  Model  for  each  Course 

Being  Sequenced 

The  first  step  in  the  sequencing  process  is  to  determine  the 
desired  terminal  behavior  of  students  being  graduated  from  each 
course  which  is  being  sequenced. 

Mastery  models  may  be  defined  in  two  ways.  They  may  be 
defined  as  a  group  of  particular  problems  which  the  student  must 
challenge  and  successfully  complete.  An  example  would  be  the 
definition  of  an  air-to-air  mission  scenario  the  student  must 
fly  containing  a  particular  mission  objective  of  controlled 
difficulty  and  a  particular  set  of  optional  contingencies  likely 
to  occur  during  the  mission.  This  type  of  mastery  model  is 
appropriate  when  complex  and  lengthy  sequences  of  behavior  are 
the  goal  of  training.  Such  behaviors,  since  they  approximate 
real-world  events  and  are  unpredictable  and  difficult  to  evaluate 
cannot  be  evaluated  exhaustively  and  separately.  The  stating  of 
a  mastery  model  as  a  problem  assumes  the  student  must  be  ready 
for  all  contingencies.  A  mastery  model  made  up  of  problems  of 
this  sort  must  be  comprehensive  and  cover  the  entire  range  of  job 
behaviors.  One  problem  situation  is  not  enough,  but  a  problem 
situation  for  each  main  area  of  job  performance,  for  instance  for 
each  main  mission  type,  may  be  required.  The  mastery  test 
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STEP  1:  DEFINE  MASTERY  MODEL 


Figure  5  -  Flow  Diagram  of  the  Syllabus  Generating  Process 


resulting  from  such  a  mastery  model  would  be  a  series  of  problem 
situations  for  the  student  to  challenge  and  complete  successfully 
to  be  fully  certified.  This  method  of  mastery  model  definition 
is  especially  important  for  certifying  more  experienced  and  com¬ 
petent  performers  because  it  samples  a  wide  range  of  behaviors 
and  calls  for  broad  skills  less  experienced  persons  often  do  not 
have.  The  difficulty  of  problems  can  be  controlled  to  create 
comparative  tests  with  a  range  of  difficulties,  and  there  may  be 
graded-dif f iculty  versions  of  one  test  which  represent  easier  or 
more  difficult  challenges.  These  may  even  be  used  to  define 
various  levels  of  certification. 

A  second  variety  of  mastery  model  definition  is  more  appro¬ 
priate  for  courses  whose  goal  is  basic,  less  comprehensive  and 
less  complex  skills.  IT  consists  of  evaluating  students  for 
mastery  in  individual  skills,  allowing  them  to  certify  in  iso¬ 
lated  areas  of  job  performance  one  at  a  time.  A  mastery  model  of 
this  sort  might  evaluate  the  student  on  some  level  of  proficiency 
at  performing  basic  skills  such  as  takeoff,  basic  intercept  or 
delivery  types,  or  refueling  under  a  given  set  of  conditions. 

The  principles  behind  both  these  forms  of  mastery  model  are 
the  same:  The  criterion  which  the  student  must  meet  is  stated. 
The  difference  in  mastery  model  types  exists  in  the  length  and 
complexity  of  the  skills  being  evaluated  and  therefore  the  real¬ 
ism  of  performance  compared  to  real  job  settings  and  the  control 
the  evaluator  can  exercise  over  the  course  of  the  evaluation 
problem.  For  a  "problem  situation"  mastery  model,  although  the 
beginning  conditions  of  the  problem  may  be  stated  very  clearly, 
subsequent  events  during  the  problem  cannot  be  carefully 
controlled.  In  the  second  variety  of  mastery  model,  which  will 
be  called  the  "individual  task"  model,  the  behaviors  to  be 
performed,  the  conditions  existing  during  performance,  and  the 
standards  by  which  they  will  be  evaluated  can  be  carefully 
designated . 

Even  though  the  mastery  model  types  are  described  here 
separately  for  purposes  of  clarity,  it  should  be  apparent  that 
they  are,  infact,  different  points  along  the  same  dimension. 
Mastery  models  of  both  kinds  must  be  constructed  for  F-16 
training.  The  individual  task  mastery  model  will  be  used  for 
earlier  phases  of  training,  particularly  to  determine  graduation 
from  RTU  and  within  phases  for  RTU  training.  The  more  comprehen¬ 
sive  problem  situation  mastery  model  will  be  used  to  measure  the 
attainment  of  later,  career-level  performance  goals.  As  the  F-16 
student  advances  out  of  one  type  of  mastery  model,  he  will  be 
progressively  advancing  into  the  other,  higher-level,  mastery 
model.  Problem-related  mastery  models  will  take  the  form:  (1)  a 
mission  to  be  executed,  (2)  a  set  of  environmental  conditions 
(weather) ,  (3)  a  set  of  threat  conditions  (some  known  to  the 
student  and  some  not) ,  (4)  relevant  mission  planning  data,  and 
(5)  a  statement  of  t’  e  expected  student  behavior,  stated  in  terms 
of  advantage  gained,  kills,  avoidance  of  kill,  or  any  combination 
of  these,  along  with  a  percentage  figure  stating  how  often  the 


student  must  be  successful  against  known  and  unknown  adversary 
attacks.  Problem  situation  mastery  models  are  likely  to  be  of 
major  importance  to  continuation  training.  Definition  of  these 
mastery  model  problems  will  take  place  in  project  Phase  V  when 
the  continuation  training  syllabus  is  generated. 

For  RTU  training  and  the  training  directly  following  it,  the 
individual  task  mastery  model  is  to  be  used  to  bring  the  student 
to  the  point  of  readiness  for  more  lengthy  problems.  The  deriva¬ 
tion  of  the  task-related  mastery  model  is  somewhat  more  mechan¬ 
ical.  First,  major  areas  of  the  mastery  model  are  defined  within 
which  competance  will  be  judged.  Then  individual  task  listing 
tasks  are  assigned  to  each  emphasis  area.  Then,  for  each  task 
the  various  appropriate  levels  of  performance  to  be  required  of 
students  are  defined.  The  final  mastery  model  consists  of  a 
statement  of  the  tasks  to  be  evaluated  by  the  mastery  model  and 
the  level  of  competance  required  of  students  for  each  task.  Each 
step  in  individual  task  mastery  model  construction  is  discussed 
separately  below. 


5. 3. 1.1  Step  1A:  Define  Major  Emphasis  Areas  and  Assign 

Tasks  to  Each 

The  main  purpose  of  defining  major  areas  of  emphasis  is  to 
bring  together  from  various  parts  of  the  task  listing  those  tasks 
which  are  likely  to  be  trained  within  one  time  frame.  This 
process  is  useful  because  it  begins  to  divide  the  entire  task 
listing  into  smaller,  more  workable  units.  It  may  be,  at  some 
future  stage  of  the  syllabus  building  process,  that  a  developer 
will  detect  some  benefit  to  be  gained  from  breaking  apart  an 
emphasis  area  and  spreading  it  across  the  syllabus.  Creative 
division  of  traditional  or  most  readily  apparent  emphasis  areas 
should  be  encouraged  in  the  development  of  innovative  syllabus 
patterns.  In  the  absence  of  such  innovation,  a  good  starting 
point  is  the  traditional  emphasis  area  division. 

Tasks  should  be  allocated  to  an  emphasis  area  which  (1) 
requires  the  same  level  of  sophistication  in  aircraft  operation 
and  (2)  requires  the  same  knowledge  base  to  be  referenced.  Tra¬ 
ditional  groupings  of  tasks  into  emphasis  areas  may  be  called 
"Basic  Flight  Maneuvers,"  "Navigation,"  and  "Air-to-Air  Combat." 


5. 3. 1.2  Step  IB:  Break  Emphasis  Areas  into  Sub-areas  and 

Divide  Tasks  Accordingly 

Further  breakdown  of  the  emphasis  areas  into  sub-areas 
facilitates  later  steps  in  syllabus  building.  This  xs 
accomplished  by  identifying  within  emphasis  areas  sub-groupings 
of  tasks  which  (1)  require  about  the  same  level  of  performance 
sophistication,  (2)  use  the  same  aircraft  subsystems,  or  (3) 
require  the  same  knowledge  base.  These  breakdowns  may  go  so  far 
as  to  specify  individual  areas  of  qualification  for  students, 
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that  is,  aircraft  operation  areas  within  which  a  student  may  be 
certified  competent,  for  example  "takeoff,"  or  "day  refueling." 


5. 3. 1.3  Step  1C:  Assign  Performance  Criteria  to  Each  Task 

for  Each  Level  of  Competence  to  be 
Certified 

As  the  final  part  of  mastery  model  development,  the  level  of 
proficiency  expected  for  each  emphasis  area  must  be  set.  These 
may  correspond  with  CRO  standards.  Intermediate  levels  of 
criteria  may  be  set  as  well  to  be  used  as  an  intermediate  level 
of  certification.  Setting  these  levels  of  proficiency  fixes  the 
sequence  of  performance  levels  through  which  a  student  must  pass 
while  progressing  toward  mastery  model  attainment.  As  soon  as 
the  student  is  capable  of  performing  a  given  area  of  task  to  a 
given  level  of  proficiency,  he  may  be  certified  at  that  level  and 
may  begin  to  challenge  the  next  level.  This  not  only  helps  in 
student  motivation  by  giving  the  student  a  measure  of  personal 
progress  and  goals  attained;  it  also  provides  a  tool  to  training 
administrators  that  allows  certifications  to  be  made  in  exact 
terms.  Designations  such  as  "mission  ready"  or  "combat  ready" 
may  be  defined  in  terms  of  a  given  group  of  tasks  performed  at  a 
given  level  of  proficiency. 


5.3.2  Step  2:  Identify  Prerequisite  Sequences  Within  Each 

Emphasis  Area 

Within  each  emphasis  area  it  is  possible  to  determine  tasks 
which  must  be  trained  before  other  tasks  in  the  same  area.  It  is 
also  possible  to  determine  which  tasks  are  independent  of  each 
other  and  may  be  taught  in  any  sequence.  This  information  may  be 
obtained  in  the  task  listing,  through  subject-matter  expert 
consultation,  and  through  judgment  only  if  necessary.  The  pro¬ 
duct  of  this  step  will  be  a  time  phasing  chart  for  each  emphasis 
are  a  showing  the  order  in  which  tasks  may  be  trained  within  the 
area,  that  is,  those  which  must  be  trained  before  others,  and 
those  which  may  be  trained  at  any  time. 

One  task  should  be  taught  before  another: 

1.  If  it  is  a  simpler,  easier-to-learn  version  of  the  same 
task  (e.g..  Normal  take-off  should  be  trained  before 
maximum  gross  wight  take-off)  . 

2.  If  the  second  task  includes  performance  of  the  first 
task  within  it.  This  should  have  been  determined  during 
the  task  listing  process  and  shown  in  the  task  listing. 

3.  If  there  is  some  reason  to  believe  that  practice  of  the 
first  task  will  make  it  easier  to  learn  the  second  task. 
There  may  be  many  reasons  for  this:  Common  controls  may 
be  used,  the  task  may  contain  common  or  similar  steps, 
or  there  may  be  other  similarities. 
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The  sequence  determined  during  this  step  within  each  area 
will  facilitate  later  steps  by  identifying  necessary  prerequisite 
relations  units  which  may  be  dealt  with  as  one  group  of  tasks 
rather  than  as  several  individual  tasks.  Care  should  be  exer¬ 
cised  not  to  identify  prerequisite  sequences  which  are  really  not 
sequences . 

A  chart  should  be  constructed  during  this  step  on  graph 
paper  as  illustrated  in  Figure  6.  The  vertical  line  to  the  right 
is  a  terminal  point  from  which  sequencing  may  begin  and  build 
backward.  In  building  sequences,  as  an  aid  to  eliminate  much 
writing  and  changing,  it  is  suggested  that  a  table  top  be  used 
and  3x5  cards  (one  for  each  task)  to  construct  the  chart.  When 
a  final  arrangement  of  the  cards  is  obtained,  the  written  version 
of  it  as  in  Figure  6  may  be  drawn. 


5.3.3  Step  3:  Combine  Area  Sequences  to  Form  General 

Syllabus  Sequence 

During  this  step  a  merging  of  the  sequences  obtained  from 
Step  2  takes  place.  The  identification  of  prerequisite  sequences 
should  begin  using  the  procedure  used  for  the  within-area  sequen¬ 
cing  activity.  As  in  that  procedure,  the  aim  is  to  demonstrate: 
(1)  tasks  which  may  be  taught  independently  (that  is,  for  which 
there  is  no  recommended  training  sequence)  and  (2)  tasks  which 
must  be  trained  prior  to  other  tasks.  Also  as  in  the  Step  2  pro¬ 
cedure,  caution  should  be  exercised  against  specifying  sequences 
which  are  not  necessary  or  directly  contributory  to  each  other. 
Doing  so  may  prevent  innovative  sequences  from  forming  during 
this  step. 

This  activity  should  not  require  a  mass  reshuffling  of  the 
tasks  between  emphasis  areas.  There  may,  however,  be  tasks  which 
are  appropriately  regrouped  on  an  individual  basis.  For  the  most 
part,  this  activity  should  consist  of  (1)  sequencing  the  major 
emphasis  areas  themselves  and  then  (2)  reassigning  the  individual 
tasks  as  required. 


5.3.4  Step  4:  Determine  Syllabus  Performance  Standards 

and  Simplified  Task  Versions  Where 
Necessary 

During  the  writing  of  CROs  and  in  earlier  steps  of  sequenc¬ 
ing  (specifically  during  mastery  model  construction),  multiple 
standard  levels  of  performance  may  have  been  specified  for  tasks. 
At  this  time  the  exact  standard  to  be  used  as  the  measurement 
criterion  for  each  task  in  the  syllabus  is  chosen.  Since  multi¬ 
ple  criteria  levels  may  have  been  specified  for  tasks  during 
mastery  model  preparation,  and  since  these  levels  reflect  on  the 
career-level  standards  expressed  in  the  CROs,  the  specification 
of  syllabus  criteria  for  each  task  may  be  accomplished  to  a 
general  degree  first  by  delineating  the  levels  on  tfy.e  mastery 
model  which  students  must  reach  for  each  group  of  tasks.  All 
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Figure  6  -  A  Prerequisite  Sequence  Chart  for  One  Emphasis  Area 


tasks  within  the  group  thus  are  assigned  a  criterion  level  at 
once  in  a  general  statement,  and  the  detailed  and  exact  values 
for  each  task  may  then  be  added  individually. 

Also  at  this  time  certain  tasks  may  be  detected  which  (1) 
are  too  dangerous  to  be  safely  performed  in  the  training  environ¬ 
ment,  (2)  are  too  costly  to  be  performed  in  the  training  environ¬ 
ment,  (3)  are  sophisticated  beyond  the  level  of  capability  a 
student  will  have  reached  during  this  time  in  the  syllabus,  or 
are  difficult  tasks  which  require  additional  cued  and  prompted 
practice  under  conditions  and  given  aids  not  found  in  the  task 
listing  version  of  the  task.  For  too  dangerou  tasks  a  form  of 
the  task  is  substituted  which  is  not  as  dangerous  but  which 
satisfies  the  syllabus  developer  that  the  student  will  have  had 
sufficient  experience  in  performance  of  a  similar  task  to  be  able 
to  perform  the  real  task  when  it  is  called  for  in  a  job  environ¬ 
ment.  For  too  costly  tasks,  less  costly  versions  of  the  same 
task  or  less  costly  similar  tasks  are  substituted.  For  tasks 
requiring  sophisticated  levels  of  behavior,  tasks  containing 
behaviors  the  student  can  perform  may  be  substituted,  or  provi¬ 
sions  may  be  made  for  instructors  to  supply  the  additional  needed 
capabilitiy  for  students  during  performance  (for  example  through 
the  use  of  a  dual-seat  aircraft).  For  tasks  requiring  special 
guided  practice,  additional  tasks  are  inserted  in  the  appropriate 
prerequisite  positions  which  will  prepare  the  students  for 
unguided  practice.  In  some  cases,  the  above  problems  will  have 
been  noted  during  task  analysis,  and  the  task  supplied  to  satisfy 
the  syllabus  requirement  may  have  been  added  and  marked  with  a 
(T)  ,  indicating  a  "training  only”  task.  New  tasks  arrived  at 
during  this  step  should  be  added  to  the  task  listing  with  such  a 
mark  to  distinguish  it  as  a  training  task  rather  than  a  job  ta'k. 


5.3.5  Step  5:  Fill  Out  Demonstration/Practice/Evaluation 

Data  For  Each  Task 

Steps  up  to  this  point  have  determined  the  gross  structure 
of  the  syllabus.  During  this  step  data  are  collected  on  the 
amount  demonstrated,  practice,  and  evaluation  required  for  the 
training  of  each  task  to  syllabus  criterion  levels  in  the  train¬ 
ing  device  to  be  used  so  that  the  exact  dimensions  of  the  sylla¬ 
bus  may  be  determined.  This  step  of  sequencing  requires  exactly 
the  same  data  to  be  gathered  as  Step  30  in  the  media  selection 
(training  device  design)  procedure.  (See  project  report  no.  20, 
"Media  Selection  and  Utilization  Plan  Report.")  The  same  data 
may  be  used  for  both  media  selection  and  this  step  of  sequencing. 
Briefly,  those  data  consist  of  an  estimate  of  the  number  of 
repetition  in  each  training  device  of  each  task  which  the  rater 
feels  students  will  require  to  attain  criterion  performance 
levels.  Data  collected  should  represent  the  best  collective 
judgment  of  instructor-level  subject  matter  experts  available. 

It  is  important  that  those  subject  matter  experts  have  had  exper¬ 
ience  in  actually  training  students  in  aircrew  training  tasks,  if 
not  in  the  same  aircraft,  then  in  a  comparable  aircraft.  During 
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the  syllabus  revision  process  provision  will  be  made  for  adjust¬ 
ing  these  intial  estimates  in  the  light  of  actual  data  gathered 
as  students  use  the  syllabus.  These  adjustments  will  either 
lengthen  the  amount  of  time  that  students  are  allowed  to  practice 
or  reduce  it  appropriately. 

As  a  quality  control  during  this  step  it  is  advised  that  the 
widest  possible  range  of  subject  matter  experts  have  an  opportu¬ 
nity  to  review  and  comment  of  the  estimates  made.  If  errors  are 
made,  they  should  tend  to  be  generous  rather  than  stingy.  As 
students  use  the  syllabus  they  will  suffer  from  too  few  opportu¬ 
nities  to  practice  but  not  from  too  many,  and  unneeded  practice 
can  be  cut  more  easily  than  extra  practice  can  be  added. 


5.3.6  Step  6:  Prepare  the  Training  Device  Syllabus  Time 

Line 

Up  to  this  point  desired  syllabus  sequences  have  appeared  as 
linear  sequences  of  tasks.  Once  training  device  practice  data 
have  been  collected  in  the  previous  step,  the  specification  of 
actual  syllabus  events,  aircraft,  and  simulator  flights  may  take 
place . 

The  formal  sequence  in  this  structure  is  the  order  of  air¬ 
craft  flight,  then  training  device  flights  in  reverse  order  of 
complexity.  Actual  time  available  for  instructional  use  during 
each  flight  is  blocked  out  on  a  chart.  Training  events  are  then 
selected  and  placed  on  the  chart,  filling  a  space  proportional  to 
the  time  they  will  occupy  during  the  flight.  This  information 
may  be  gathered  from  the  data  collected  in  Step  5.  That  data 
will  also  indicate  how  many  sequential  flights  will  require  the 
practice  of  a  given  task  and  will  suggest  the  number  of  repeti¬ 
tions  needed  during  each.  Without  violating  any  of  the  prerequi¬ 
site  sequences  set  in  earlier  steps,  all  tasks  are  assigned  to 
flights.  This  may  in  many  cases  become  a  fitting  process,  fill¬ 
ing  spaces  of  unused  flight  time  with  independent  and  moveable 
tasks . 


The 
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following  potential 
this  sequence: 


problems  should  be  watched  for  in 
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Flights  should  not  be  filled  so  full  of  activities  as 
prevent  the  normal  flow  of  discussion  and  feedback 
between  student  and  instructor. 


to 


2  Adequate  re-try  time  should  be  available  in  each  flight 
to  provide  for  unexpected  contingencies  without  disrupt¬ 
ing  the  program  of  the  flight  and  forcing  some  activi¬ 
ties  out. 


3. 


Activities  during  a  flight  should  alternate 
possible  between  difficult  and/or  new  activ 
more  simple  and/or  familiar  ones.  This  wil 
excessive  mistakes  due  to  overstress  on  the 
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5.3.7  Step  7:  Insert  Academic  Instruction  for  Each 

Training  Device  Event 

Sequence  information  contained  in  the  objectives  hierarchies 
is  used  during  this  step  to  insert  academic  training  just  prior 
to  the  event  during  which  that  academic  training  will  be  exer¬ 
cised.  Objectives  w  ;  ^  nrpremi  i  site  to  a  given  task  should 

be  placed  in  sequence  prior  to  the  performance  of  the  task. 

In  all  cases  the  question  should  be  asked  continually 
whether  all  prerequisites  are  being  identified,  and  a  regular 
survey  of  the  entire  body  of  objectives  to  spot  possible  omis¬ 
sions  is  recommended. 

The  arrangement  of  academic  instruction  in  the  syllabus 
should  show  awareness  of  (1)  time  and  motivation  factors  and  (2) 
topic  similarity  relationships.  Time  and  motivation  factors 
should  capitalize  on  the  average  expected  length  of  the  student 
study  time  and  arrange  the  amount  of  instruction  in  a  lesson  or 
unit  such  that  progress  measurable  to  the  student  may  be  achieved 
during  each  one.  Topic  similarity  relationships  should  be  used 
to  group  academic  instruction  lessons  or  units.  This  may  produce 
lessons  covering  only  one  type  of  procedure  (e.g.,  emergencies) 
or  one  phase  of  flight  (e.g.,  takeoff  related  procedures). 
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